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ABSTRACT : 

A methodology is developed for auditing cost effectiveness 
analyses of major technological changes. The methodology 
is applied to the Work In Process Inventory Control System 
(WIPICS) recently implemented at NARF, North Island. The 
approach involves using data on NARF operations to estimate 
cost functions for each major program of the NARF both 
before and after the change. Cost comparisons using these 
models do not show a clear cost savings for the WIPICS 
system. 
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and in terms of a practical use on a particular technological change. 
Since major portions of the report concern data and specific circum- 
stances associated with the practical example, it is useful to detail 
the particular technological change involved at this point, 

B, AN EXAMPLE OF TECHNOLOGICAL CHANGE 

The Naval Air Research Facility at NAS North Island (NARFNI) is one 
of seven naval air rework facilities throughout the continental United 
States responsible for major maintenance, conversion and repair of United 
States Navy and Marine Corps aircraft and related components. To 
accomplish this mission NARFNI employes approximately 6800 civilian 
workers and spends $150 million on annual operating expenses. 

Maintenance of equipment is accomplished under one of three primary 
programs, with two of the programs receiving inputs either directly from 
the customer or from the third program (see Fig, 1), 
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FIGURE 1. NARFNI Repair Programs 
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these programs work is further assigned to one of the six 
of the Production Department identified in Figure 2, 
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A. INTRODUCTION 

It is commonly accepted that technological advancements are 
arriving at a faster and faster pace with the state of the art generally 
far ahead of the functional application. One reason for this gap is 
the costly enterprise of transfer ing theory into practice. When the 
initial transfer is made a natural question to ask prior to extensive 
implementation is **Will it be cost-effective?** Current methods of 
cost-effectiveness analysis are tedious and usually employ a detailed 
analysis of the process which incorporated the change. Such analyses 
are inherently imprecise due to the possibilities of overlooked depen- 
dencies, double counting of costs, and external changes which may affect 
the effectiveness of the process and yet not be accounted for in the 
analysis. Thus when a cost-effectiveness analysis is completed a 
reviewing agency might well ask, **Are the conclusions of the analysis 
correct?*' As mentioned, the methods employed in the analysis make 
this second question a difficult one to answer. 

Nevertheless, it is important to answer the above question. Many 
technological changes are implemented on a prototype basis. Therefore, 
it is increasingly important that a careful audit of the cost-effective- 
ness analysis be made. That is, the costs, cost savings, and increased 
effectiveness attributable to the technological change must be carefully 
documented after the installation of the prototype and before similar 
changes are implemented at other installations. 

This report provides several alternative methods of auditing cost- 
effectiveness analyses. It compares the methods on a theoretical basis 
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The assignment of work to various programs and production divisions 
can be thought of as a **job shop** operation in which each shop performs 
a specific task. Thus an item entering rework is subject to being 
dismantled, its subassemblies distributed for rework, and then re- 
assembled prior to returning to the customer. This type of operation 
places emphasis on the scheduling of rework in that ideally the shops 
are completing work on related parts in the same sequence to reduce the 
total repair time of the major end items. Scheduling is also important 
to the shop managers who must order material and plan personnel assign- 
ments based on the projected arrival of work from other shops. This 
entire scheduling problem is intimately related to the technological 
change to be described in the following section. 

NARFNI is primarily funded by the Navy Industrial Fund (NIF) 
which provides a working capital fund to finance repair operations. 

The finished product is returned with a bill to the customer for the 
work done. This **debt** is then paid by transferring funds from the 
customer’s appropriated maintenance funds to the NARF’s working capital 
fund. Under this concept NARFNI is required to control its finances 
to incur zero profit at the end of each fiscal year. This control is 
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exercised at Quarterly Planning Conferences in which representatives from 
the customers and NARFdl meet to plan the following quarter’s work load 
input and prices for the work to be accomplished. l^ARFIII is reimbursed 
under basically two types of contracts, cost reimbursable and fixed price, 
the majority of the work being performed under fixed price contracts 
established at the Quarterly Planning Conference* This type of contract 
gives NARFI'JI incentive to minimize costs subject to the required work load, 
a fact which is instrumental to the auditing method investigated. 

In January 1972, NARFNI installed an industrial information system, 
the Work in Process Inventory Control System (WIPICS) , to assist in the 
location and scheduling of inventory in the rework process. WIPICS was 
requested and initiated by the Management Systems Development Office (MSDO) 
in an effort to assist the NARF’s in performing their mission at a minimum 
cost to the government. If the prototype at NARFNI is determined to be 
cost-effective WIPICS will then be installed at the remaining six NARF’s. 
Its major milestone are listed in Figure 3. 



1. Early 1969 

2. Late 1969 

3. Early 1970 

4. July 1970 

5. Late 1970 



ROHR Corporation studied NARFNI 
MSDO requested contract 
NAVAIRSYSCOM evaluation and approval 
SECNAV authority for prototype 
Contract negotiated 
Contract date, D-Day 
WIPICS developed 
WIPICS test 

WIPICS prototype started 



6. 4 January 1971 

7. D + 9.5 months 

8. D -f 11 months 

9. D + 11.5 months 



FIGURE 3. WIPICS Milestone 
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WIPICS consists of a central computer which is linked to the job 
shops by 164 ” touch tone" telephones, 20 alphanumeric terminals, and 
four teletypewriters. In addition, the system is equipped with an 80 
word audio-response unit to allow two-way communication through the 
telephones. When an item first arrives at NARFNI it is broken down 
into identifiable subassemblies. For each of these subassemblies a 
computer record is created and entered into WIPICS. This record con- 
tains information such as identification, required work, required 
material, location, and status. Each job shop inputs arrival and de- 
parture information through the "touch-tone" telephones. Also, an 
unexpected delay in work due to lack of material or some other short- 
coming is input ed as it occurs. At the completion of the work day a 
print-out of this information is produced to aid shop managers in 
projecting the rate and mix of work that will be entering their respective 
shops in the near future. 

The inauguration of the WIPICS prototype was accomplished by 
creating computer records for all jobs presently in the shop to preclude 
a lengthy start up before evaluation could begin. Simultaneously, the 
operational costs accountable to WIPICS were transferred as overhead to 
be applied to the price of jobs in shop. Thus, during January, 1972, 
NARFNI initiated a technological change which was to be subjected to 
cost-effectiveness analysis and, at the same time, provided a suitable 
situation to use in investigating a method for auditing cost-effective- 
ness analyses. 

C. PREVIOUS WORK 

This report documents our research in auditing cost-effectiveness 



analyses and summarizes several previous reports, presentations and 
theses generated by the research project. In particular six master's 
theses in Operation Analysis references [1, 2, 3, 4, 5, 6] provide 
much of the detail and data that underlie the report. A brief synopsis 
of each follows. 

Spooner [1] details the general methodology to 
be followed in using continuous production functions 
of the Cobb-Douglas variety to model a multiproduct 
facility before and after a technological change. 

He also provides the initial before WIPICS data 
on engine and air frame jobs and a computer program 
to convert this infoirmation to a flow of work over 
time at NARFNI. 

Myers [2] developes linear economic models of the 
air frame and engine programs at NARFNI. He also 
provides an estimation procedure for determining 
the transformation matrix from data on jobs. 

Bradley [3] uses data provided by Spooner to 
estimate production functions for airframes, 
engines, and F/J components. He also does some 
preliminary analysis of the timing of fluctuation 
in workload at the NARF. 

Trafton [4] extends Bradley's work by estimating 
cost functions of the same form as the latter 
production functions. He also develops and uses 
an estimation procedure for the cost function 
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derived from the Constant Elasticity of Substitution 
Production Function. 

Tye [5] continues this line of reasoning by 
estimating similar functions for sets of data 
collected after WIPICS has been implemented. He 
tests their significance, constructs prediction 
intervals and derives the first preliminary con- 
clusions concerning the cost-effectiveness of the 
system. 

Finally McGarrahan [6] develops models similar to 
Myers’ for the after WIPICS data and compares 
the results from these models to those derived 
by Tye. He also reports a third type of model and 
compares it to the other two types. 

The above theses contain data, computer programs, and specific 
results which support the information provided in this report. 

The report is divided into several sections. The section below 
presents the methodology of the study and the alternative mathematical 
models that are employed. Section three describes the data that were 
used and various transformations and adjustments to that data. In 
section four the various estimation procedures that are employed are 
presented. The results of those procedures are reported in section five 
along with tests of hypotheses, and prediction intervals. Section six 
of the report contains comparisons of the various models, the conclusions 
of the study, and topics for further research. 
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II METHODOLOGY AND SCOPE OF THE PROJECT 

A. INTRODUCTION 

Usually the major source of effectiveness for new equipment or pro- 
cedures is in terms of factor saving. That is, an analysis will frequently 
document expected effectiveness in term of man-days saved or decreases in 

wasted raw materials or replacement of several more expensive pieces of 
equipment. 

Other contributions to effectiveness are more difficult to estimate 
and evaluate. However, some attempt is usually made to include effects 
like increased speed of production, higher quality of output, and more 
control over the production process. 

Frequently, an attempt is made to assign dollar values to each of 
these measures of effectiveness and a cost-effective change is defined 
as one with a greater value of effectiveness than its costs. 

Frequently unstated in the analysis is the assumption that these 
effects are expected only if nothing else changes. 

After the equipment has been installed or the procedure implemented, 
one can gather data on the changes that have taken place. For example, 
we know what has happened to factor usage, we can measure the new speed 
of the production process, and perhaps we can determine quality changes 
in output. Here the important questions are not what changes have taken 
place, but why the changes have occurred. In particular are the changes 
due to our new equipment or procedure ? 

Notice that after the fact, many other things may have changed as 
well. In particular, the outputs of the organization may have changed 
in both quantity and type, prices of factors, many have changed or new 
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constraints may have been placed on the organization or old ones relaxed. 
Finally, the manager of the enterprise may not have acted exactly as the 
cost-effectiveness analysis expected. For example, suppose a new piece 
of equipment replaces old equipment and several men, instead of laying 
off the men the manager re-assigns them as trouble shooters. As a 
result, output increases dramatically with no increase in costs. Here 
costs did not decline as expected, instead the manager chose to increase 
output. 

Current methods of auditing cost-effectiveness analyses involve 
looking at factors by categories and determining if their usage has 
changed. A portion of the change in factor usage is directly related 
to the new equipment by verbal argument and a dollar value is assigned 
to that portion of the change in factor usage- Thus the relation between 
the factor savings and the new equipment frequently is a verbal argument 
constructed by an outside observer. These allowed effects are then 
compared to costs. Other effects like quality changes are handled 

separately. 

Our proposal for auditing CE analyses recognizes that an outside 
observer cannot effectively trace second order effects though a massive 
enterprise. Thus the proposal is to look at aggregated summary measures 
of the organization’s behavior in each of several areas before and after 
the technological change. 

In particular our approach to determining the effectiveness of 
WIPICS is to develop a model of the production behavior of the NARF both 
before and after the implementation of the system. These production 
models are then compared for several sets of circumstances and conclusions 
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are drawn from the comparisons* Three different production models have 
been used for these purposes and each of them is described below* 

B. THE CONTINUOUS PRODUCTION FUNCTION OF A MAJOR PROGRAM 



A production function is frequently used to describe the relations 
between factors and the homogeneous output that is produced by their use. 
Ordinarily, the function describes this relation for a specified period 
of time. In addition the function typically possesses the following 
characteristics : 

1. W = f(Z]^,Z 2 ,Z 3 ) where W is output, Zj^,Z 2 ,Z 3 are quantities 
of factors used, and f is a single valued, continuous function 
whose first and second derivatives exist, f represents the 
maximum output that can be produced with a given vector of 



2 . 



factors. 
3f 



3Z^ 

tive. 



> 0, the marginal product of each of the factors is possi- 



3. The matrix 






az^azj 



is negative semidef inite. This requires 
the rate at which one factor may be substituted for another 
while producing a given quantity of output diminishes as the 
second factor is substituted for the first. 

To elaborate on this last property, it may be possible to produce 
a PAR Repair on an A3B with input combinations one and two below, but 
one would hardly expect that the possibilities of substituting labor for 



material at the same rate would extend to situation three. 



Situations 



Variables 


1 


2 


3 


PAR Repairs 
A3B 


1 


1 


1 


Direct Man 
Hours 


12,600 


12,000 


10,200 


Material 

Costs 


11,400 


15,000 


25,800 


FIGURE 


4. Production Tradeoffs 





By estimating a production function for an entire program we are 
not involved with modeling the detailed relations among shops or the 
exact relation between WIPICS and the cost producing variables (factors) 
or the effectiveness generating variables (outputs). This eliminates 
the need to conduct expensive and time consuming measurements of many 
variables and also prevents the possible double counting problems that 
are difficult to avoid in a cost-effectiveness analysis. 

On the other hand, the production function is sufficiently detailed 
to permit the analysis of the NARF’s opportunities for tradeoffs among 
factors. For example, the trade-off between man-hours and repairable 
airframes given some level of output and material usage is measured by the 
production function. Thus, the opportunity to take cost savings due to 
WIPICS in the form of reduced man hours or reduced inventory of repairable 
airframes is measured. 

The basic information embodied in the production function may be 
combined with some additional assumptions concerning the organization 
to determine a cost function for a program. 



For example, it may be assumed that the NARF attempts to minimize 
the cost of producing a given quantity of output from the program. In 
addition, if the NARF has no control over the prices it pays for factor 
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then a cost function may be derived from the program: 

Min C = Cq + ^1^1 ^ 2^2 ^ 3^3 (1) 

s.t. W = f (Z]^ , Z2> ^3) 

as, C = Cq + g(W, Pj^, P2, P3) (2) 

In the above program is fixed cost; P^, P^, and P^ are the 

prices of factor; and W is a measure of output for the program. 

It is important to notice that the costs of production are thus 
related to several variables which influence the environment of the organ- 
ization. The relation between program costs and prices and outputs is 
thus captured. 

Two functions of the same form as g may be estimated by econometric 
techniques one each for the before WIPICS program and the after WIPICS 
program. Finally, the hypothesis that these functions are significantly 
different from one another may be tested for some level of confidence 
and we may determine if the relation which describes costs in the program, 
has changed significantly with the implementation of the WIPICS system. 

Assuming the cost functions are different from one another, they 
may be used to describe changes in the situation after the technological 
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change. Whether the changes are beneficial or not, may depend on the 
environment of the enterprise at a particular point in time. For 
example, Figure 5 shows projection of two cost functions on the cost, 
quantity axes. Suppose function B describes the before WIPICS 
situation and function A the after situation. Then whether the tech- 
nological change is beneficial or not, depends on the expected level 
of output of the program. That is, the cost functions allow one to 
compare the before and after situation at alternative levels of output 
and the conclusions may very well depend on the level of output chosen. 




FIGURE 5. Illustrative Cost Curves 

This approach to auditing cost-effectiveness analyses of tech- 
nological change will permit one to compare before and after situations 
even if other things do change. In particular, changes in output and 
and prices of factors are automatically accounted for in the model. In 
addition any action which the manager of the organization takes to mini- 
mize the cost of producing a given level of output is compatible with the 
model and consistant with a valid interpretation of the results. 



Finally, anticipated changes in the constraints on the manager 
or his enterprise may be incorporated into the programming problem 
at (1) and these changes also may be handled if data on the constraints 
are collected and used in the estimation procedure. 

Two alternative production functions have been used in the study 
so far. The first of these is the Cobb-Douglas production function: 

“1 ^1 yi 
Wi = AiZiiZ3^22i3 

“2 ^2 ^2 
W2 = A2Z2XZ22Z23 



Here and measure output of programs one and two respectively. 
Solving the programming problems at (2) yields: 

c = + Cl + 



where 



Cl = AlWi 



ai+Si+Yi 



C2 ^2^2 



“ 2 '^^ 2'*”'^2 “ 2 '''® 2‘'”'^2 



and and are constants. The properties of production functions 

require that: 



A* > 0, A* > 0 
0 < ^ ^ 



The second production function used in the study is the C. E. S. 
production function. Here an additional specification is imposed, that 
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one of the inputs is determined by forces outside the organization’s 
control so that there are only two decision variables in the production 
function. Here: 



'^1 “ 

Now letting C’ = + P3(Z^3+Z23) 

may be solved to yield. 





1 

Q 


(1-6i)Zi2 ] 




”^2 


-0,/p 


(1-62)222 ! 





a programming problem similar to ( 2 ) 



= r’ 



+ C, + C. 



where 



/^ ll \ /^ l \^ ""^1 ^^^1 
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The properties of production functions require that the C.E.S. parameters 
conform to the following restrictions; 

j O2 » » Y2 ^ ® ^ 

and ^l’‘^2 ^ 

It may be noted in passing that either production function may be 
used to investigate returns to scale. For the Cobb-Douglas production 
function decreasing, constant or increasing returns to scale obtain as 
a + 6 -h Y is less than, equal to, or greater than one respectively. 

For the C.E.S. production function decreasing, constant, or increasing 
returns to scale obtain as a is less than, equal to, or greater than 
one respectively. 

Finally, it may be noted that the C.E.S. function is much more 
general that the Cobb-Douglas as it approaches the latter as a special 
case as p— >-0. 

C. THE CONTINUOUS PRODUCTION FUNCTION FOR A JOB 

This production model is very similar to the above model. It does, 
however, overcome one troublesome assumption above, namely the require- 
ment that the output of a program be homogeneous. Here the production 
function is applied to a job defined as a particular type of repair on 
a particular airframe or engine. Since the time to complete the same 
job may vary over the period analysed, the number of days a particular 
job was in shop is introduced as an explicit variable in this case. 

These cost functions for each job type may be derived and the 
parameters estimated. The functions may then be estimated again for 
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the after WIPICS situation. The two sets of functions may then be 
compared directly as above. Also, the total costs for some representa- 
tive distribution of work load may be predicted by each as a further 
test of significant change between the before and after WIPICS situation. 
This production model for a job enjoys similar characteristics to the 
production model for a major program. 

The mathematical model used for the continuous job production func- 
tion is also a production function of the Cobb-Douglas variety. Here 
however, the function applied only to a single job and as a result out- 
put was measured by a constant (W^j), where the subscripts refer to 
the type of engine or airframe being reworked and the type of repair 
being done respectively. This has the effect of including a dummy 
variable in the cost function associated with the job and for all prac- 
tical purposes makes it impossible to measure returns to scale for 
either program. The detailed cost functions used are discussed in 
section four of the report. The advantage of this model is that it 
does not require a homogeneous output measure to be defined. The 
disadvantage is that it requires more data and more judgement in deter- 
mining an appropriate distribution of workload for comparison. In 
addition, if a job occurs in the before situation that is not represented 
in the after situations or vice versa, the effect on this job cannot 
be compared. 

D. THE LINEAR ECONOMIC MODEL 

Linear programming is a mathematical technique for solving constrained 
optimization problems of a special type. A linear economic model is a 
specific type of linear program consisting of the maximization (or 
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minimization) of a linear function of n variables subject to m linear 
inequality constraints. Linear programs (and their close mathematical 
relative the input-output models) have become increasingly important 
to microeconomic theory in recent years. 

In the linear economic model being constructed it is necessary 
to define a process. A linear production process is an activity by which 
one or more outputs are produced in fixed proportions by the application 
of one or more input factors in fixed proportions. In this case there 
is at least one process per job type. The production process as defined 
is homogeneous of degree one which implies constant returns to scale. 

For example, if all inputs to a process are doubled then the output will 
also be doubled. A linear production function is formed from a collec- 
tion of linear production processes that may be used simultaneously. 

The optimal solution to a linear economic model as described above 
consists of finding the combination of m processes from n available 
processes such that a linear objective function is maximized (or 
minimized) . 

The following assumptions are made in the formulation of a linear 
economic model: 

(1) The estimated processes (reworking of engines and aircraft) 

are linear functions and therefore these processes exhibit constant returns 
to scale. 

(2) The above linear processes may be estimated by the aggregation 
of a finite set of observations over some time period. 

(3) The management objective of the NARF will be assumed to be 
minimization of costs subject to completion of all work demanded by the 
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operational forces of the Navy. 

(4) Prices used in the model are constant and may be estimated 
from the production data furnished by the NARF. 

To present the model, it is necessary to have some mathematical 
notation available. Definitions will be as follows: 



m = 
n = 



R = 
P = 



Z = 



T = 



number of output constraints 
number of production processes 
(n > m) 

column vector of available resources = 



R1 

R2 

R3 



column vector of prices associated with Rl, R2 , 
respectively 

column vector of activity levels = 

i l,...,n 

Technology matrix of observed processes at NARF 
[(itri-3) X n matrix] 



and R3 




where 



T^ = units of output 
(mxn matrix) 



10000 ...0 

0110 ...0 

00010 ...0 



and 

T^ = resource matrix 



0 . 



■ FJ 

i = 1, 2, 3 



0 1 



th 

with = amount of i — resource used per unit 



th 

output of j — activity. 
Y = column vector of production 
output desired 

X l|...^m. 



[^] 



= diagonal matrix of number of days in shop 
for each process per unit of output 

C = column vector of penalty costs = 

X l^...^n 



o ■■ 



fi] 



where c^ = penalty cost in dollars incurred for each day in shop 

t h 

associated with the i — process. 

Now that a basic mathematical vocabulary exists for the variables, 
processes and costs, the linear economic model will be presented and 
discussed : 

objective function; minimize + C^T^Z 



-T, 



-Y 

"“r“ 



constraints: 

subject to: 

The objective function consists of two terms: 

T 

P *T2*Z = actual cost in dollars of resources used for activity 

level vector Z 
T 

C • Z = penalty cost in dollars for activity level vector Z. 

The constraint set may also be separated into two parts for discussion: 



-T^* Z ^ -Y is a set consisting of the first m rows of 



the constraints which force the activity level vector 
Z to choose a set of processes which satisfy the 



production vector Y. 

^ R consists of the last 3 rows of the constraint set and 
ensures that actual resources used do not exceed 
available resources* 

It is clear that the size of the resource vector R is highly 
dependent on the production vector Y, In fact, these two vectors must 
be chosen carefully to avoid an infeasible linear program. The easiest 
approach to avoiding an infeasible linear program is to make the re- 
source vector R very large, therefore the optimal solution to the 
model will not be constrained by an active resource constraint. In 
economic terms, this means that the "customer" is willing to pay the 
NARF as much as necessary to accomplish the work specified by production 
vector Y. If the NARF is assumed to be efficiently operated, then 
the costs incurred will be minimized. 

In summary, the objective function of the model will be a minimum 
when the model is not constrained by the resource vector R. In other 
words, the costs are minimum when the resource constraints are inactive 
constraints. The value of the objective function represents the total 
cost incurred to accomplish production vector Y based on past perfor- 
mance data of the NARF. 

It should be noted that the linear economic model presented in this 
chapter is not a production management tool in the sense that the model 
chooses "processes" by which the NARF should rework aircraft or engines. 
The model only provides a budget cost plus a penalty cost for a specified 
amount of work to be done. Some information is available from the model 
concerning tradeoff values among the three resources. The model can also 
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determine the minimum amount of any single resource required to accomplish 
the specified work. 

As with the previous models, the linear economic model is estimated 
from both before and after WIPICS data and the two resulting models may 
be compared process by process and for a representative workload at the 
NARF. The data requirements of this production model approximate those 
of model two and the estimation procedure is not as well developed in 
this area as it is for the previous two models. In addition, the assump- 
tion of constant returns to scale and a small number of discrete ways to 
combine resources are restrictive. Nevertheless, the linear economic 
model does preserve more detail about the production process at the NARF 
and for this reason it is used and compared to the previous models. It 
also permits changes in prices, and the level and distribution of output 
requirements to directly affect costs in a program. 
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III. DATA AND ITS USE 
A. DESCRIPTION OF AVAILABLE DATA 

The preceding mathematical production models are defined in terms 
of physical measures of inputs and outputs and well defined prices for 
these quantities. In fact, data on these variables cannot be obtained 
in most large organizations and NARFNl is no exception. The data obtained 
from NARFNl consists of information on 837 aircraft and 1865 engine re- 
works performed during the period March 1970 to March 1973. For each 
rework the following information was available: 



(1) 


Type of engine or aircraft 




(2) 


Identification number 




(3) 


Type of work done 




(4) 


Induction date 




(5) 


Production date 




(6) 


Production load norm (man-hours) 


(NORM) 


(7) 


Airframe change man-hours 




(8) 


Direct labor hours expended 


(DMHR) 


(9) 


Direct labor cost 


(DLB$) 


(10) 


Direct material cost 


(DML$) 


(11) 


Applied overhead cost 


(D0H$) 


(12) 


Navy Industrial Fund (NIF) rate 


(NIFR) 



The manner in which the above statistics are accumulated and used by NARF 
North Island are enumerated in [7] 

In order to keypunch the basic data it was necessary to assign codes 
for different types of engines/aircraft and the types of work done. These 
codes are listed in Tables 1 and II. The coded data is listed for reference 
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purposes in Spooner [1] and McGarrahan [6]. 

One additional item of information concerns a penalty cost assigned 
to aircraft and engine down time at the NARF. These penalty costs per 
day were calculated from average flyaway unit procurement costs for each 
engine and aircraft* The particular penalty costs used are reported by 
McGarrahan [6]* 

These data were used to form several proxies for the variables of 
Section Two* A brief description of each follows: 
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TABLE I 





Codes Assigned to 


the Engine Program 




Engine Type 


Code 


Engine Type 


Code 


T58-GE-1 (A/F) 


51 


T64-GE-3 (A/F) 


63 


T58-GE-3 (A/F) 


53 


T64-GE-6B 


65 


T58-GE-5 (A/F) 


55 


T64-GE-6B (PAWN) 


66 


T56-A-8P 


56 


T64-GE-7 (A/F) 


67 


T58-GE-5 (C/G) 


57 


T64-GE-413 


69 


J57-P-AA/22 


71 


T58-GE-8F 


81 


J57-P-10 


72 


T58-GE-8B 


82 


J57-P-20A 


73 


T58-GE-8B/F 


83 


J57-P-22 


74 


T58-GE-8B (C/G) 


84 


J57-P-420 


75 


T58-GE-8B (HH2) 


85 






T58-GE-8B/F (CONV) 


86 


J79-GE-8B 


91 


T58-GE-10 


89 


J79-GE-8B/C 


92 






J79-GE-8B (RDTE) 




* Abbreviations are defined 


J79-GE-10 


95 


as follows 








OVHL = overhaul 




Work Type* 


Code 


CONV = conversion 




OVHL 


01 


PAR = Planned aircraft 






repair 




OVHL/CONV 


02 


SUP = Supply 




PAR/REP 


03 


REP = Repair 




SUP /REP 


04 


SEA = South East 


Asia 


SUP/REP/CONV 


05 


CONUS = Continental United 



States 
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TABLE II 

Codes Assigned to the Aircraft Program 



Aircraft Type 


Code 


Aircraft Type 


Code 


C-2A 


10 


Ch-3B 


31 


E-2A/B 


11 


RH-3A 


32 






SH-3A 


33 


F-4J 


21 


SH-3A/G 


34 


F-4B 


22 


SH-3D 


35 


F-4G/B 


23 


CH-46A 


41 


F-8J 


25 


CH-46D 


42 


F-8H 


26 


CH-46F 


43 


RF-8G 


27 


UH-46A 


44 






UH-46D 


45 






CH-53A 


48 






CH-53D 


49 


Work Type* 


Code 






OVHL 


01 






PAR 


02 






PAR/CONV 


03 






PAR/MOD 


04 






PAR/MOD/REP 


05 






PAR/REP 


06 






PAR/ SEA 


07 






PAR/CONUS 


08 







*See Table I for Work Type abbreviations. 



1> Output , Physical measures of output of the NARF would require 
hundreds of variables on numbers of aircraft and engines of various types 
with several different kinds of repair work done. Furthermore, there is 
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no obvious common denominator of the repair work. So many different 
measures of output would be extremely difficult to handle and would 
require many more observations to produce statistical significance. 

Hence, a proxy variable of production load norm, N, was chosen to 
measure output. Production load norm has the advantage that across jobs 
it is measured in the same units, expected man-hours required to complete 
the job. In addition, it is, in many cases, what determines the cost to 
the fleet of having the repairs made. That is, the fixed price of a 
job is negotiated as two components, the production load norm, as a 
measure of the difficulty of the job, and the expected NIF rate, as the 
expected cost per man hour on the job. These might usefully be thought 
of as measures of the quantity of repairs to be completed and the price 
per unit of the repairs. 

2. Input one - Labor . Direct man-hours expended, L, are used 
as a measure of labor used. This variable excludes indirect man-hours 
and fails to distinguish among alternative types of labor used. In 
spite of these drawbacks, it is the most accurate of the proxy variables. 

3. Price of input one - Wage rate . The ratio of direct labor 
costs, L$, to direct man-hfiurs expended, L, is used as a proxy for 
the wage rate P . Because this variable is averaged over all types of 
labor it may be influenced by factors other than a general increase 

in wages. For example, an airframe that requires very technical work 
and uses high priced labor exclusively will be associated with higher 
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values of this proxy. Also, periods of time that involve large quantities 
of overtime labor will show high values of the variable. Still, this 
variable is thought to be a reasonably accurate measure of the wage rate. 

4. Input two - Items to be repaired . It seems clear that if out- 
put is to be a measure of repairs on engines or airframes then one input 
must be repairable engines or airframes. Unfortunately, these items 
have many different descriptions and are in various states of disrepair. 

As a crude proxy for this variable the unweighted number of items in 
ship at any point in time, I, is used as a measure of items to be 
repaired. 

5. Price of input two - Penalty cost per item . The NARF does not 
pay for items to be repaired nor does it provide ”loaners” to the 
fleet while the repairs are being undertaken. Nevertheless, from the 
point of view of the Navy this pipeline of repairable items is costly 
and both the NARF and the Navy are concerned with reducing the size of 
the pipeline. Hence, the models that we are using include as costs of 
the NARF a penalty cost for each repairable item held by the NARF on any 
day. Our proxy measure of penalty cost per item per day is procurement 
cost of the item divided by expected service life, P^. This measure 
tends to overstate the true costs because it ignores scrap value of the 
item. However, this is more than compensated for by its understatement 
of cost due to ignoring the time value of money. The particular penalty 
costs used are listed in McGarrahan [6]. 

6. Input three - Material . Because many small items of material 
and parts are consumed no attempt has been made to construct a quantity 
index of material. Instead, the dollar value of material consumed, M$, 
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is used as a proxy for this variable. 

Price of input three . Since the quantity of material is 
measured in dollars the price of material is defined to be one. Over 
time periods long enough for inflation to be a problem an appropriate 
price index would be used for the price of material. 

In addition to the definition of proxy variables, one further 
adjustment was required to use the available data from NARF NORIS to 
estimate production and cost functions. The available data consisted 
of observations on expenditures for each job with different jobs lasting 
varying lengths of time. On the other hand, the continuous program 
production functions describe the entire program over some given length 
of time. Thus, the data were aggregated in the following manner: For 

each job, the values of norm, man-hours, labor cost, material cost, over- 
head cost and penalty cost were prorated equally over the days for which 
that job was in the shop. Then, for each day in the period under con- 
sideration, the prorated values were totalled for all jobs which were 
in the shop on that day. The result of this aggregation is that for each 
day in the period under consideration, we have an estimate of the total 
number of man hours (L) used on that day, the total costs of labor 
(L$) for that day, the total cost of materials (M$) for that day, the 
total cost of overhead (0$) for that day, the total penalty cost (P$) 
incurred on that day, and the total hours of norm (N) produced on that 
day. In addition, the number of jobs in the shop (I) on each day was 
tabulated. All of the prorating and aggregating was done separately 
for the two programs — aircraft and engines. A computer program to per- 
form this task is included in Spooner [1] . 
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Table III. Correlation Coefficients for Engine Raw Data 
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Table IV. Correlation Coefficients for Aircraft Raw Data 
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These aggregated data were used as described in section A to estimate 
the parameters of the cost and production models, 

B. PRELIMINARY DATA ANALYSIS 

As a first step in determining the relationships among the raw data 
elements, the matrix of correlation coefficients was calculated for each 
program. These results for the engine and airframe programs are 
reported in Table III and IV, respectively. 

It is perhaps important to note the relatively low correlation be- 
tween direct material cost and any of the other variables in both programs. 
This is one possible indication of the fact that material costs are thought 
to be the least reliable of the data obtained. It is also interesting 
to note the relation of number of days in shop to the other variables. 

In the shorter engine program, these correlations are relatively low 
while they are significantly higher for the longer aircraft program. 

In addition, the aggregated data was used to get a preliminary view 
of the flow of work over time at the NARF. Conversations with NARFNI 
personnel indicated that it was their opinion that induction rates were 
fairly constant, independent of time, while production rates were low 
at the beginning of a quarter and high at the end of a quarter. The 
average daily induction and production rates were computed, using a 
moving block average. 

These rates were computed from both the aircraft and engine programs 
in terms of hours of norm arriving or leaving per day, actual man hours 
of work arriving or leaving per day, and actual physical units arriving 
or leaving per day, (Figure 6 and Figure 7). The results were not sen- 
sitive to changes in the size of the moving window in the range of seven 
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ENGINE INDUCTION AND PRODUCTION RATES 
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to twenty-one days. These plots tended to confirm NARFNI personnel in 
their beliefs about the production and induction rates over time. Pro- 
duction did increase toward the end of a quarter and did drop off sharply 
at the beginning of a quarter. The induction rate did have some varia- 
tion over time, but it was about one-fifth as great as the variation of 
the production rate. 

Number of jobs in shop versus time was plotted to confirm this 
relationship between induction rate and production rate. Number of jobs 
in shop should be increasing at the beginning of a quarter when produc- 
tion rate is less than induction rate and should be decreasing at the 
end of the quarter when production rate is greater than induction rate 
(Figure 8). This plot of jobs in shop versus time, with a maximum near 
mid-quarter, is consistent with the production rate and induction rate 
relationship described above. 

The variable, number of jobs in shop (I), was considered to be 
descriptive of the production and induction rates and, incidentally, 
also descriptive of shop congestion. By using number of jobs in shop 
as an independent variable in the Cobb-Douglas production function, it 
was possible at once to describe efficiencies due to specialization of 
personnel, inefficiencies due to shop congestion, and work flow through 
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NUMBER OF JOBS IN SHOP 
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IV PROCEDURES FOR ESTIMATION OF 
MODEL PARAMETERS 

A. THE LINEAR ECONOMIC MODEL 

To formulate a linear economic model it is necessary to define 
a process. A process consists of a mathematical description of the 
amount of resources used (i.e. labor, material, etc.) to produce a unit 
amount of an output measure. The problem is how are the raw data obser- 
vations used to estimate a process. The most natural way would be to 
estimate a process as an average amount of resources required to do a 
specific type of work on a certain engine or aircraft. This procedure 
may not be desirable because there may be alternate processes available 
to produce the same output. For example, to overhaul an F-8J it may 
be possible to use a **normal** amount of man-hours and have a low material 
cost. An alternate process may exist whereby the same work may be 
accomplished by spending more money on material and reducing the man- 
hours required. The essence of a linear economic model is, then, to 
identify these alternate processes and to use linear programming to 
select the most efficient processes to operate. 

In an attempt to identify these alternate methods within the basic 
data observations a clustering algorithm was developed based on the 
Hierarchical Clustering Strategy of Lance and Williams [8], The general 
procedure is to select a group of observations containing several different 
types of engines (or aircraft) and some different types of work within each 
engine group. These observations are then analyzed using the clustering 
algorithm for similarity of the vector of input/output measures. Vectors 
which are similar will use input resources in approximately the same 
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fixed proportion for a unit output measure. Similar observations can 
then be averaged to yield a ’’process”. 

Several trial selections were made with both engine and aircraft 
data. The engine data was observed to cluster groups of the same engine 
type together with a high degree of consistency. For examplei if 20 J-79 
engines were selected as part of a trial run of 100 observations, a 
typical result would be for 17 or 18 of the original 20 to be clustered 
in the same group. However, the trial selection with the aircraft data 
were not nearly so consistent. The cluster analysis showed the aircraft 
data had wide variations among the input variables. The clusterings 
observed were highly irregular and contained a mixture of different 
aircraft and different work. Since the aircraft clusters which resulted 
from this preliminary analysis were not easily interpretable as ’’processes”, 
it was necessary to do some preliminary classification before using thfe 
clustering algorithm. In particular, it seemed ridiculous to have processes 
which involved radically different job types even if they seemed to use 
resources in the same proportion, so a decision was made to form processes 
consisting of only similar engines (or similar aircraft) with identical 
types of work. 

For the engine data, similar engine/work types were grouped together 

within each job order. The resulting raw data deck was then ordered by 

engine type, job type and calendar quarter in which the work was done 

(in special cases where a very few observations were available the engine/ 

job types were not separated into calendar quarters). The average of the 

observations in each calendar quarter then provided an aggregated obser- 
vation on each engine^ work type worked on by the NARF during that quarter. 



This observation was taken to be a potential process for that engine/ 
work type. At this point there were 47 different engine/work types 
and 103 processes by which the work could be done. A computer program 
was used to examine the dominance relationships among all processes with 
the same engine/work type. This computer program is described in Appen- 
dix G of Myers [2]. The dominance relationships among processes are very 
important in a linear program. For example, if process 1 and process 
2 produce the same output but process 2 uses more of every required 
resource than process 1, then process 1 dominates process 2. In a linear 
program that minimizes costs, process 1 will always be selected as pre- 
ferred to process 2. Therefore, if the linear programming model is to 
have alternate processes that reflect choices to be made then the domi- 
nance must be eliminated between all processes with the same output. 

The dominance program was run with the 103 processes as previously 
determined based on job order numbers. The dominance relationship 
between each pair of processes that produced the same output was examined. 
If no dominance existed then that group of alternate processes was left 
alone. If dominance existed among any of the processes in the group then 
a subjective decision was made to combine two of the processes. For 
example, if there were five processes and dominance existed between three 
pairs of them, then two processes would be selected to be combined. The 
decision on which pair to combine was entirely subjective. The group 
would then have four alternate processes remaining. The dominance com- 
puter program would then be run again and the remaining alternate processes 
would be examined for dominance. This procedure was repeated until all 
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dominance was eliminated within process groups that had the same output 
(same engine/work type). Since the outputs were intended to be measured 
separately (as opposed to an aggregate output measure) it was not ne- 
cessary to examine the dominance between processes of different outputs. 
The original 103 potential processes were aggregated down to 81 processes 
(on the 47 engine/work type) by this dominance removal procedure. The 
final 81 processes were used as input for the Linear Economic Model on 
engines described earlier. 

The aircraft data consisted of 365 raw data observations. Con- 
sidering each type of aircraft and type of work as a separate output 
meant that 70 outputs were required. This small amount of data then 
had to be used to estimate not less than 70 distinct processes. A 
decision was made to estimate only a single process for each output be- 
cause of the limited data available. In this case there were no alternate 
processes so it was not necessary to check for dominance. The linear 
program in this case would not have a choice among processes and there- 
fore the solution becomes trivial. 

An important extension of this work would be to refine methods for 
identifying alternate processes in such small data bases. Computer 
programs for the data analyses described above can be found in appendices 
B - G of Myers [2]. The resulting linear programs for the engine and 
aircraft model will be compared to other models and across the Before 
WIPICS - After WIPICS period in Section VI. 

B. THE CONTINUOUS PRODUCTION FUNCTION FOR A PROGRAM 

Three alternative econometric models based on the two production 
functions detailed in section II are derived in this section. Each 
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involves different specifications about how random variables enter the 
cost and production function and hence different assumptions concerning 
the behavior of decision makers at NARF North Island. Throughout this 
section the subscripts referring to the engine and airframe programs 
have been dropped. Each model is applied to both these programs. 

Model one. The first and simplest of the three models assumes that 
the relation among inputs and output at the NARF is of the form of the 
Cobb-Douglas production function. Additionally, it is assumed that an 
unobservable error term enters the relation in a multiplicative fashion 

so that the production function itself is stochastic. Substituting the 

c . 

proxy variables and including e as the random error yields; 

N. = A l“ M$T e ^ 

1 1 i 1 

where i indicates the time period of the observation. Taking natural 
logarithms of both sides yields: 

£n (N^) - £n A + a iln + 6 iln 1^ *f Y M$^ + 

This equation is the econometric model associated with model one and is 
estimated by linear regression techniques. The results are reported in 
section V. 

Model two. The second model also makes use of the Cobb-Douglas 
production function. Here, however the production function itself is 
assumed to hold with certainty. Random errors enter the model in the 
process of solving the cost minimization programming problem. Since 
the exact form of the production function is not known the planned 
quantities of inputs required to produce a given quantity of output may 
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linearity of the C.E.S. cost function it is estimated in two stages 
instead of one. The first stage involves the choice of the relative 
quantities of the two inputs, labor and the number of items in shop. 

This choice is thought to be subject to a random error and is determined 
by: 



L. 

1 




(1-6)P 



£.i 



6P 



li 



-1/p+l 



e 



e 



1 



Taking natural logarithms of both sides yields: 



. 8 q + = 



where 



= £n 
^1 " 



6o - ^ 






This equation is the first stage of the econometric model associated with 
model three and is estimated by linear regression techniques. 

Let the estimates derived for parameters in the first stage be b^ 

A 

and bj^ respectively so that ^ ^l^li ^ prediction of y^. 

Also estimates of 6 and p may be derived as: 



6 



1 

gbo/bi + 1 



and 



Finally 



e 



bo+b^Xi 



e 





is a prediction of 




Substituting 
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differ from those actually required. This leads to build ups or short- 
ages in inventories of inputs and thus to random errors in the cost 
function. See Trafton [4, pp 19-20] and Dhrymes [9, pp. 232-234] for a 
complete derivation of this model. 

Substituting the proxy variables and including the random error 
term yields: 

C = pa/a+ 6 +y 6 /a+ 6 +y ^i 

i i i li ® 

where 

= L$ + M$^ + P$ 

Taking natural logarithms of both sides yields: 

£n C. = 8 q + 8 ^ An N. + In P^. + 83 £n P^^ + 

where 

= 1/a+B+Y 3 q = A 

= a/a+B+y 

B 3 = 3/a+B+y 

This equation is the econometric model associated with model two and is 
estimated by linear regression techniques. The results are reported in 
section V. 

Model three. Model three differs from the previous two models in 
two regards. It is based on the C.E.S. production function instead of 
the Cobb-Douglas function but it is also different in that material cost 
is no longer an input to the production process. Because of the non- 



these estimates into the C.E.S. cost function and supplying an error 



term yields: 
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1 1 




L^i- 


LvJ 
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e + 1 e +1 





l+b. 



1/p 



where 



C. = L$ + P$. 
1 i 



This equation may be reexpressed as: 



y! = iln N, + elX. + e! 



where 




giving the second stage of the econometric model associated with model 
three and it is also estimated by linear regression. The results of 
model three are also reported below. 
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C. NONLINEAR PROGRAMMING FOR C.E.S. ESTIMATION 

Because the two stage estimation procedure for the C.E.S. function 
described above produces parameter estimates whose properties are not 
known, it was considered desirable to investigate alternate estimation 
strategies. In particular, any least squares parameter estimation pro- 
blem can be formulated as a nonlinear optimization problem. If the 
model is y = f(X, 3) with independent variables X, dependent variable 
y, and parameters 3 to be estimated, from the n observations 
(Xi, y^) i = 1,..., n, then the least squares solution is 3 which 
solves the NLP. 

I (y^ - f (Xj* 6))^ 

If f(X, 6) is linear in B, then this is an unconstrained quadratic 
optimization for which standard regression produces a closed form solution. 
Otherwise, as in the case of the C.E.S. function, iterative nonlinear 
programming methods for gradually approaching the solution can be used. 

Since the two stage estimation procedure sometimes yields parameters 
which have intuitively ridiculous signs, it may be desirable to also 
incorporate constraints (such as — 0) into the nonlinear program. 

This approach to estimating C.E.S. cost function coefficients was in- 
vestigated in some detail. The C.E.S. cost function involves the 
model 

c = [(1 - 5) + 6 (I/Lfp] + PI (I/L)] c 

where economic theory imposes the following restrictions on the parameters 

o>0, Y2:0> 0<6<1 
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By transforming the variables we can incorporate these constraints into 

2 2 2 

the optimization problem. If we let a ^ , y ^ , p * -1, 

2 

6 = (sin z^) then for any value of the parameters z^ i«l, 2, 3, 4, 

the parameters a, y, p and 6 will satisfy the above. Hence the least 
squares optimization problem can be formulated as the unconstrained 
minimization of a function of the four parameters z^, z^. 

This minimization problem was investigated for the before WIPICS air- 
craft program. 400 before WIPICS observations (Julian dates 0191 - 1225) 
were chosen as test data. Tye [5] reports the results of the two stage 
C.E.S. estimation procedure applied to the same data. The 400 observa- 
tions were split into two subgroups consisting of the first 300 and last 
100 observations so that the stability of the cost function could also be 
investigated. The unconstrained minimization of the least squares func- 
tion was accomplished by the Conjugate Gradient method due to Fletcher 
and Reeves [10]. The minimizations were performed for 300, 100 and all 
400 observations. In each case several different starting points were 
chosen including the parameter estimate from the 2-stage procedure, to 
test for unimodality of the least squares objective function. The 
results of this procedure were not promising and hence will not be re- 
ported here in detail. The following qualitative remarks will indicate 
some of the problems encountered: 

1. Numerical difficulties were severe for the highly nonlinear 
C.E.S. cost function. Typical runs started with the least squares func- 
tion value (sum of squared residuals) from 10^^ to 10^^. Final values 

9 

when the minimization terminated were on the order of 10 . Gradient 
components were also frequently extremely large in magnitude. As a 
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result abnormal terminations due to numerical problems such as computer 
overflow or underflow were encountered even in double precision. Careful 
scaling of the variables was necessary to get any results at all. 

2. The two stage C.E.S. estimates were not anywhere close to least 

30 

squares solutions, having objectives in the range 10 . When started 

at these estimates, the nonlinear programming procedure reduced the 

9 

objective quickly to about 10 . 

3. The solutions obtained when the nonlinear minimization was started 
from different initial estimates had objective function values which 

were similar, but the parameter estimates obtained, displayed substantial 
variability. This indicates that there are many alternative parameter 
estimates all of which are approximately equivalent under the least squares 
criterion. We do not have other theoretical properties for these esti- 
mates (in particular their statistical properties are unknown) so the 
choice of which parameter estimates to use is not simple. 

Due to these difficulties, it was decided not to pursue this line 
of investigation farther. The two estimation methods tried (2 stage, 
nonlinear programming) for the C.E.S. cost function gave substantially 
different results and neither has known properties. Thus, for the bulk 
of our work, the simpler Cobb-Douglas Production and Cost Functions will 
be used. The Cobb-Douglas functions are linear in the parameters and 
hence ordinary regression gives parameter estimates with known statistical 
properties . 

D. THE CONTINUOUS COST FUNCTION FOR A JOB 

The econometric model corresponding to the production model for a 
job is similiar to model two described above. However, to facilitate 



understanding of this model a complete deviation is presented here. 
Suppose the production function for a job of type i j is of the Cobb- 
Douglas form with four inputs, i.e. 
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W. ., = A. . L. ,, T. .. M$, .. NIS, .. 
ijk ij ijk ijk ijk ijk 



here W. is a measure of repairs produced by completing job i, j, k, 
ij tc 

^ijk length of time this job is in shop, and is the 

average number of jobs in the shop while this job is being worked on. 

is assumed not to be under the control of the NARF for any par- 
ticular job and hence is not a decision variable in the following costs 
minimization. Also, suppose that the NARF attempts to minimize the cost 
of producing any given job so that its goal is to 



^ijk ^Oijk ^Jlijk ^ijk ^ijk ^^ijk 



s.t. W, = A, . L. ,, T. ,, M$. NIS, 

ijk ij ijk Ijk ijk ijk 



As in model two errors are expected in the attempt to perfectly meet 
this goal and this results in a stochastic cost function of the form 



a+e+Y a+e+Y 



C. - C„. = A, . W. 

ijk Oijk ij ijk 



iijk 



S 

a+S+Y 

li 



NIS 



a+3+Y 

ijk 



cijk 



Two alternative econometric models are formulated from the above 
equation. The first of these models is based on the assumption that out- 
put depends merely on the type of aircraft being repaired and the classifi- 



cation of repairs. Thus two par reworks on an F4-J would be assigned the 
same measure of output. Thus 



5A 



”ijk = (K = 1, .... n) 



In this case, the variable is indistinguishable from the constant 

and also the variable is unable to be distinguished from either 

of the others. Hence, all three variables must be combined into dummy 
variables that vary only with job description. 

The first econometric model is based therefore on 



a+6+y a+B+Y 

C. NIS e 

ijk Oijk ij ilijk ijk 



eijk 



or taking natural logarithms and redefining variables 



y.., = 2,n + n ^n Pn.., + v In NIS.,, + 

^ijk ij ^lijk ijk ijk 



where 



y - In (C , • - 1 ,^ 

^ijk ijk Oijk 



^ ^a+3+Y a+B-by 

B. . = A, . W. . P^, 

ij ij iJ 



1 

aH 
P. 



n = a/a+B+y 



V = 6/a+B+y 



Another model formed from the above equation is due to a different measure 
of output. Here output is supposed to be proportinate to production load 
norm but the factor of proportionality depends on the job description thus 
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'^ijk ■ ^ij ”ijk 



In this case also other variables must be combined with the constant term 
to form a viable model and the resulting form of the model is 



where 






yijk ' <‘=nk - 



e 



* a+6+Y a+a+Y 

D. . = A. a. . 
ij i]k 13 li 



= 1/a+a+Y 



6^ = a/a+a+Y 



The results of some comparisons using this kind of cost function for 
a job are reported in section VI below. An alternate derivation of some 
similar models is given by McGarrahan [6], 
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V ESTIMATION AND TESTING OF THE CONTINUOUS MODELS 
A. PROCEDURE 

This section reports the results of applying regression analysis 
to estimate the coefficients of the three continous models described in 
Chapter IV. 

1. Cobb-Douglas Production Function 

log N = + 6^ log L + log M$ -f 6^ log I + e 

2. Cobb-Douglas Cost Function 

log C = Bq + log N + B2 log P^ + 33 log P^ + e 

where 



C = total cost = L$ + M$ + P$ 
P^ = price of labor = L$/L 



P^ = penalty price of items in shop = P$/I 



3. C.E.S. Cost Function 



£n(L/l) = Bq 




e 



for stage one, and 



£n C 



- )Pi 



+ P. 



(t)( ■ tn N + 6' tn 



1-b, 



bn/b 






+ 1 

e -r 1 






for stage 2. 
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Recall that for the C. E. S. cost function, total cost C does not 
include material cost. For each of these models the coefficients were 
estimated for several different sets of data both before and after 
WIPICS and for both the engine and aircraft programs. 

First, a period of 400 days of data well before the installation 
of WIPICS was chosen. This data was split into two sections composed 
of the first 300 and last 100 days, and the models were fit to both 
parts to investigate the stability of the production and cost function 
coefficients. Since the 400 day period was well before the implemen- 
tation of WIPICS, the results obtained should reveal some of the pro- 
blems to look for in the estimation, that is, the expected variability 
due to factors other than WIPICS. 

Second, data periods immediately prior to WIPICS and after 
installation and breakin of WIPICS were used to again estimate the 
coefficients. The resulting BW/AW models will be compared in section 
VI to see if WIPICS had an impact on the estimated costs of the aircraft 
and engine programs. 

The results of these estimations and the stability tests will be 
reported in this section. 

B. RESULTS FOR THE BEFORE WIPICS PERIOD 

400 consecutive days of before WIPICS data covering the Julian dates 
0191 - 1225 for aircraft and 0215 - 1250 for engines were split into 
the first 300 and last 100 observations using the regression procedures 
outlined in section IV, the three models were estimated for this data. 

The resulting coefficients are shown in tables V through X. The standard 
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COBB-DOUGLAS PRODUCTION FUNCTION - ENGINES 
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COBB-DOUGLAS COST FUNCTION - ENGINES 
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error of each estimated coefficient is shown in parentheses below the 
2 

coefficient, R is the fraction of the variance of the dependent 
variable explained by the regression equation, SE is the standard 

A. 

error of the dependent variable, and p is an estimate of the serial 
correlation of the random error terms in the model. 

An intuitive examination shows that the coefficients do not look 
very similar between the 300 and 100 day data samples. A statistical 
measure of this difference can be obtained using a statistical technique 
known as the Chow test [11, pp 192-207], Briefly, the Chow test is as 
follows: 

Let 

= the sum of squared residuals from the regression on 
all 400 observations, 

= (the sum of squared residuals from the regression on 
the first 300 observations) 

+ 

(the sum of squared residuals from the regression on 
the last 100 observations). 




k = the number of coefficients estimated in each regression. 
Then under the hypothesis that the coefficients are the same in all three 
regressions, the ratio 

Q3/k 

^ " Q^/ (400-2K) 
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is distributed as an F statistic with k and 400 - 2k degrees of 
freedom. If F > 400-2k) , then reject the hypothesis of equality 

of the coefficients at the e confidence level. 

The Chow tests were performed on the Cobb-Douglas production and 
cost functions for both aircraft and engine programs. Details of the 
tests are given in Table XI. For all cases, the hypothesis that the 
coefficients were essentially the same for all 3 regressions was soundly 
rejected. This indicates that during the period considered, the co- 
efficients of the regression models were not completely stable. Since 
the data on which these regressions are performed is aggregated time 
series data, it is wise to test for autocorrelation of the residuals. 

This was done for the Cobb-Douglas cost functions (since they are the 
models we will use the most), and in both the aircraft and engine pro- 
grams the estimated serial correlation coefficients (p) were extremely 
high. The standard first order autocorrelation correction [11, p 260] 
was made for this data and the regressions were repeated on the corrected 
data with the results shown in Tables XII and XIII. 

Chow tests run on these corrected models still rejected the hypo- 
thesis that the three equations were statistically identical at the 95% 
level, but the computed F values were not nearly so large as for the 
uncorrected models. 

C. USE OF COST FUNCTIONS AS PREDICTORS 

A second approach to validation of these models involves assessing 
their value as predictors. This approach has been followed for the 
Cobb-Douglas and C.E.S. cost function models. For each of the cost 
function models, the data was again divided into the first 300 and the 
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CHOW TESTS 



Reject = if 


F > F (4,392) = 2.39 
(5%) 




aircraft 


engine 


Cobb-Douglas 
production function 








.037 


.065 


^2 


.032 + .001 


.039 + .007 


F 


11.9 


40.5 


Cobb-Douglas 
cost function 








.031 


.143 


^2 


.024 + .001 


.067 + .019 


F 


23.5 


65.0 



TABLE XI 



COBB-DOUGLAS COST FUNCTION - AIRCRAFT 
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COBB-DOUGLAS COST FUNCTION - ENGINES 
(corrected for autocorrelation) 
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last 100 observations. Cost functions were estimated by regression 
using the first 300 days data, and the resulting cost equations were 
used to predict the cost for each of the last 100 days of data. As a 
criterion for comparison, the mean squared error (MSE) was computed 
both for the first 300 days (upon which the cost equation is based) and 
for the last 100 days predicted values. 

If e^ is the error of the predicted cost for the i^^ observation 
a confidence interval can be empirically constructed by using the ratio 
of two standard errors of the function over the actual mean costs for 
the period. The result is a percent error in cost. 



% error 




X 100. 



The % error approximates + 2 standard deviations which accounts for 
about 90% of the mass in most probability distributions. Thus, there is 
approximately a 90% confidence that the predicted cost is within the 
percent error computed of the actual cost. 

This procedure was performed on the cost functions estimated from 
the 300 day period, and then on the 100 day period with costs being 
predicted by the 300 day models. The Cobb-Douglas and C.E.S. cost 
functions were also compared to the "naive" models 



C = 3N 
C = a + 3N 



which predict cost based only on norm, to determine the improvement from 
using the cost function models. The results for the Cobb-Douglas cost 
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function are given in Table XIV. The comparison indicates that the 
Cobb-Douglas Cost function does a substantially better job of both 
explaining and predicting total cost than the naive models for the 
aircraft program. For the engine program the differences among the 
three models are not so significant, and none of the models is a parti- 
cularly good predictor of total cost. 

For the C.E.S. cost function a similar comparison was made. Direct 
comparison between Cobb-Douglas and C.E.S. is not easy since the C.E.S. 
total cost does not include material cost while the Cobb-Douglas cost 
does. The results for the C.E.S. cost function are given in Table XV. 

Again, the C.E.S. cost function is superior to the naive models 
for the aircraft program, but in the engine program this is not the 
case. 

In summary, then, both the Cobb-Douglas and the C.E.S. cost 
functions do a good job of explaining and predicting total cost for 
the aircraft program. Neither Cobb-Douglas nor C.E.S. offers advantage 
over naive models for the engine program. 

A similar procedure to that of the above discussion uses statistical 
methods outlined in Theil [12, pp 134-135] to construct prediction 
intervals. Let C’ represent the estimated cost and accept the assump- 
tions of the standard linear model. Then the following probability 
statement holds: 



P(£vi C| - H < £n C^ < 5.n C^ + H) = 1-a 




where 



aircraft 


engines 


Cobb-Douglas 




cost function 




first 300 days + 4.0% 


+ 7.0% 


last 100 days + 5,0% 


+ 15.7% 


C = B • N 




first 300 days + 11.2% 


+ 8.0% 


last 100 days + 9.0% 


+ 17.9% 


C = a + BN 




first 300 days + 11.1% 


+ 7.3% 


last 100 days 4* 8.4% 


+ 18.0% 







2 \J MSE expressed as percent of actual cost. 



TABLE XIV 
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Aircraft 


Engines 


C.E.S. cost function 






first 300 days 


+ 2.3% 


+ 11.1% 


last 100 days 


+ 3.1% 


+ IA.6% 


o 

II 






first 300 days 


+ A. 8% 


+ 10.0% 


last 100 days 


+ 7.2% 


+ 13.7% 


C = a + 3N 






first 300 days 


+ 4.8% 


+ 9.4% 


last 100 days 


+ 6.6% 


+ 14.7% 



2\/mSE expressed as a percent of actual cost. 

(Note total cost for this table does not include material cost.) 



TABLE XV 



73 



X = matrix of independent variables used to 
construct the model 

Vq= vector of independent variables used to 
predict ilnC^ 
t^^2 ” t-statistic 

This interval is in logarithms. The transformed interval in costs 
becomes non-symetric and non linear in the independent variable of V^. 

Thus, it is more accurate to compare the extreme widths of the intervals 
than the mean values compared in subsection 1. 

Prediction intervals were computed for the 100 day period using the 
300 day models. Of the 100 prediction intervals resulting the minimum ^ 
and maximum interval were transformed to dollars and then divided by 
the mean cost for the 100 day period to obtain a maximum and minimum width 
as a percentage of the average cost (parenthetical entries of Table XVI) . 
Finally, the percentage of actual costs that fell outside their correspond- 
ing prediction interval is reported in the last column of Table XVI. 



Airframes 



Engines 





Max. 


Min. 


% Out- 


Model 


Interval 


Interval side Interval 




6729 


5365 


4% 


CD 


(8.5%) 


(6.8%) 




CES 


4480 


3347 


32% 


(4.0%) 


(3.0%) 




CD 


5375 


3783 


62% 


(12.5%) 


(8.8%) 




CES 


6943 


4836 


29% 


(20.5%) 


(14.3%) 





TABLE XVI Extreme Widths of 90% Prediction 
Intervals (in dollars) 
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The uneven prediction results between the cost functions may be 
indicative of the Chow Test’s conclusion that the two periods of data, 

300 vs, the following 100 days, are in fact different and a model of one 
period is a poor prediction of costs in another period. This possibility 
is important because the periods are not separated by a recognizable 
technological change. It reemphasizes the need to include the accuracy 
of the models in the final comparison of estimated costs, 

D. ESTIMATED MODELS FOR THE BEFORE WIPICS - AFTER WIPICS COMPARISON 
For the before WIPICS - after WIPICS comparison the decision was 
made to use only the Cobb-Douglas cost function corrected for auto- 
correlation. The primary reason for this, is that autocorrelation is 
clearly a problem for data of this type. Straightforward corrections 
for autocorrelation exist for linear regression models (such as the 
Cobb-Douglas cost function) but the effect of autocorrelation on non- 
linear models (such as the C.E.S, cost function) is not known. 

For these regressions the time periods were chosen as close as 
possible to the technological change so the before and after situations 
would be as similar as possible except for the WIPICS system. The obser- 
vations used are: 



Aircraft 



Number of Observations 



Julian dates 



Before WIPICS 



400 observations 



0316 to 1350 



After WIPICS 



90 observations 



2090 to 2179 



Engines 



Before WIPICS 



400 observations 



0247 to 1281 



After WIPICS 



160 observations 



2158 to 2317 



75 



different periods are used for aircraft and engines since the average 
length of stay at the NARF is different for jobs in the two programs* 

The resulting Cobb-Douglas cost functions (corrected for autocorrelation) 
are presented in Tables XVII and XVIII, Comparison of these cost func- 
tions with each other and with other models will be the subject of 
section VI, 



COBB-DOUGLAS COST FUNCTION - AIRCRAFT 
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COBB-DOUGLAS COST FUNCTION - ENGINES 
(corrected for autocorrelation) 
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VI COST COMPARISONS BEFORE AND AFTER WIPICS 

A. PROCEDURE 

The cost function models described in the previous sections of this 
report were used to compare costs on typical workloads for the before 
and after WIPICS situations. The results of these cost comparisons are 
reported in this section. For each cost equation the parameters were 
estimated twice, once using before WIPICS data and once using after 
WIPICS data. The resulting two cost functions will be called the 
Before WIPICS cost function and the After WIPICS cost function. 

Applying the Before WIPICS cost function to a particular workload 
gives an estimate of what the cost of performing that work would have 
been if it had been done before WIPICS was installed. Similarly, using 
the After WIPICS cost function on the same workload gives an estimate 
of what the cost would have been if that work had been done after 
WIPICS was installed. The difference of these two cost estimates yields 
an indication of whether WIPICS was cost-saving for that particular work- 
load, Any indicated cost savings can then be compared with WIPICS costs 
to see if WIPICS is cost-effective. Most of the results reported in this 
section will deal with the aircraft program since even though WIPICS 
has been in operation for some time, it has not been directly applied 
to the engine program. For the Cobb-Douglas cost functions, the engine 
program results will also be given as an indication of possible changes 
not due to WIPICS, 

B, CONTINUOUS COST FUNCTIONS FOR A MAJOR PROGRAM (COBB-DOUGLAS) 

The Before and After WIPICS Cobb-Douglas cost functions for both 
aircraft and engines, corrected for auL'ocorrelation, were given in 
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TABLE XIX Cobb-Douglas Comparison of Average Daily Costs for 



section V - D. The actual workloads for the NARF during the Before 
and the After WIPICS periods were used as representative workloads for 
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the cost comparison. The results are given in tables XIX and XX. 

The costs indicated in the tables Include estimated cost of labor, material, 
overhead, and penalty for time in rework. Since the Cobb-Douglas cost 
functions operate on prorated aggregated daily data, the cost estimates re- 
present the average daily total cost for all jobs in the shop over the 
time period being used. 

Positive values of differences indicate a cost savings while negative 
differences indicate that performing the same work After WIPICS would 
cost more than if it had been done Before WIPICS. 

Detailed discussion of these results will be postponed to section 
VI - E, after the results from all the different cost functions are avail- 
able. 

C. LINEAR ECONOMIC MODEL COST FUNCTION 

The linear economic model of section II - D was used to estimate 
Before WIPICS and After WIPICS costs for the aircraft program. 837 air- 
craft jobs were separated into three groups: 

1. 414 jobs before WIPICS 

2. 158 jobs in a buffer period 

3. 265 jobs after WIPICS 

The buffer period consisted largely of jobs started before and finished 
after WIPICS was initiated. That period included all jobs with induction 
dates between Julian dates 1181 and 1334 inclusive. These dates were 



arbitrarily chosen to allow for the long time period required for aircraft 
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TABLE XX Cobb-Douglas Comparison of Average Daily Costs for 
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processing plus the indeterminate period while WIPICS was being initially 
tested. The before and after groups were the same as those used to 
generate the regressions used in the job cost model. 

The two groups were compared and aircraft-work categories common to 
both periods were selected for the cost comparison. The jobs in these 
20 common categories were then selected, representing 70.4% (257/365) and 
68.7% (182/265) of the original before and after WIPICS groups, respec- 
tively. In the interest of descriptive brevity, let 
I B j- be the set of observations before WIPICS 

be the subset of observations of | b| in the 20 common 
categories 

|a^ be the set of observations after WIPICS 

be the subset of |a} that matches the categories 

W\ 

Activity vectors and objective function coefficients were then estimated 
for both data sets | and using the procedures developed by 

Myers [2], The resulting linear economic models were formulated with 
three different objective functions. 

1) Min; Total Cost = [P^T^ + C^T^] Z 

T 

2) Min: Operations Cost = P T^Z 

T 

3) Min: Penalty Cost = C T^Z 

In order to compare the models, two test cases were run against both the 
before and after models. The test cases consisted of the actual jobs in 

and { ^Be two price vectors associated with them. Activity 

and price vectors run against their own models simply provide actual 



cost determinations, but the runs against the opposite model represent 
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TABLE XXI Linear Model Cost 
Total Cost - Aircraft 
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predicted costs. The costs were calculated for each of the objective 
function versions and the results are listed in Tables XXI, XXII, XXIII. 
The costs in these tables are the estimated total costs of doing all the 
jobs in the workload in question. Figures are listed in millions of 
dollars. The upper right corner prediction of Table XXI, for example, 



the before WIPICS period, it is predicted that the total cost would have 
been $40,799 million.” 

As the comparative results indicate, the after WIPICS period costs 
were estimated to be higher in each of the cost variations used. Com- 
parison of entries in the T^ matrices for before and after situations 
emphasized the increases in resources used per hour of NORM. In every 
process at least one coefficient was higher for the after WIPICS period, 
with many cases showing two and three increased coefficients. Any de- 
creases were relatively smaller in magnitude than the increases. This 
model contains no inherent inflation compensation for the materials and 
overhead expenditures such as the DMHR price has. 

Comparing the activity vectors before and after WIPICS highlighted 
another potential problem. The two vectors presented radically different 
levels of activity in many of the processes. Such differences could 
greatly magnify any deviation of the linear relation from the true 
relationship, which is more probably non-linear. The coefficients used 
in these linear models and the details of the | and { ^ } work- 



is read, ”lf the jobs represented by 




had been accomplished during 



loads can be found in McGarrahan [6]. 
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D. CONTINUOUS COST FUNCTION FOR A JOB 

The continuous cost function for a job was developed in section 
IV-D, Two versions of this cost function were estimated and compared 
using the same before and after WIPICS workloads |b|- and |a| as 
in the linear model. 

The two versions of the cost function used are: 

^2 ^3 ^4 

1) C = A. NIS N 

1 L D 

^1 ^2 ^3 ^4 

2) C = A. P^ T NIS N 

1 L 

where NIS is the average number of jobs in the shop over the time period 
when this job was in the shop, T is the length of time this job stayed 
in the shop, P^^ is production date, and all other variables are as pre- 
viously defined, 

A total of ten relationships were estimated from the data in 
and I A I , These represented before and after WIPICS characterizations 
of the following: 

1, Version 1 2, Version 2 

a. Total Cost a. Total Cost 

b. Operations Cost b. Operations Cost 

c. Penalty Cost 

The estimation was done only for the aircraft program. Details of the 
estimated coefficients and statistical properties of these models can 
be found in McGarrahan [6], The resulting cost functions were then 
applied to the jobs in data sets | } and | ^ (recall these contain 



job categories which are present both before and after WIPICS) tc obtain 
cost comparisons. The resulting comparisons are given in Tables XXIV 
through XXVIII. Figures in the tables are estimated total cost to perform 
all the jobs in the given workload. Costs are in millions of dollars. 
Positive differences indicate predicted cost savings after WIPICS, nega- 
tive differences indicate predicted cost increases after WIPICS. 

The results do not seem to indicate consistent superiority of one 
cost function over the other. Further discussion of these results will 



be included in the next section. 
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TABLE XXIV Job Cost Comparison - Version 1 - Total Cost 
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TABLE XXV Job Cost Comparison - Version 1 - Operations Cost 
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TABLE XXVI Job Cost Comparison — Version 1 - Penalty Cost 
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TABLE XXVIII Job Cost Comparison - Version 2 - Operations Cost 
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E. SUMMARY OF COST COMPARISONS 

The cost difference for the various cost functions are brought 
together in Table XXIX. The differences reported are all predicted 
co^ before WIPICS cost function minus predicted cost from after 

WIPICS cost function. For the Cobb-Douglas models the differences are 
expressed as dollars per day while for the other linear and job models 
the differences are in millions of dollars for completing the workload. 
Positive values indicate cost savings negative values indicate cost 
increases . 



AIRCRAFT COST DIFFERENTIAL COMPARISONS 
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TABLE XXIX 
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VII. SUMMARY AND CONCLUSIONS 

This report has explored several alternative approaches to the problem 
technological change. Three distinct types of models have been 
documented and several versions of some types have been examined in detail 
and applied to the problem. 

The linear economic model is the most detailed of the model types. It 
permits different relations between costs, inputs, and output for each dif- 
ferent job description. This model type has several drawbacks however. 
Estimation procedures for the model are not well developed and the statis- 
tical properties of the estimators are unknown. The model requires large 
quantities of data. It also requires the restrictions of constant returns 
to scale and independence of production processes; two assumptions which 
are unlikely to hold in any given application. 

The second model type is based on a continuous production function for 
a program. The model uses data that is aggregated over jobs and is there- 
fore not affected by the fact that individual jobs may not be independent 
of one another. One of the models, based on the Cobb-Douglas production 
function, is able to be straightforwardly estimated by statistics with well 
known properties in the absence of perverse circumstances. This model also 
has the advantage of not imposing a requirement of constant returns to scale. 

On the other hand the model is not as sensitive to changes in workload as the 
others. It also relies heavily on production load norm as a measure of out- 
put, at best a relatively imprecise measure. The advantages of using aggre- 
gated data in the model are balanced by the fact that the aggregated data 
set tends to be autocorrelated . Thus a special, though not unusual, correction 
must be applied in using the model. 
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The models based on a continuous production function for a job combine 
some of the characteristics of the previous models. These models are able 
to account for changes in the mix of jobs without imposing the restriction 
of constant return to scale. They also permit estimation by statistics 
with well known properties. The models do not use aggregated data and there- 
fore present no problem of autocorrelation. However, these models do suffer 
from the fact that each job is assumed to be independent of any other and as 
a result errors are introduced when this assumption is violated. 

Several versions of each of the model types were applied to data supplied 
by NARFNI for periods of time both before and after the introduction of the 
WIPICS system. The results of the application are reported in Chapter VI and 
briefly summarized in Table XXIX. 

The results in Table XXIX apply only to the aircraft program because the 
WIPICS system, while operating, had not been applied to the engine program 
at the time data was collected, March 1973. 

Even a brief glance at Table XXIX indicates that the results are not 
conclusive. Only the two versions of the last model type indicate cost savings 
for the WIPICS system in excess of the direct costs of the system and even here 
there is significant disagreement when applied to the before WIPICS workload. 
The other model types indicate that the costs of the system exceed the cost 
savings generated by it. 

Several general conclusions may be drawn from the study. Obviously, the 
evidence that now exists is insufficient to conclude that the WIPICS system is 
cost effective at this time. The results of the various regression analyses 
are too contradictory to warrant such a conclusion. This result compares 
quite favorably with other evidence which indicates that the system at this 
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time had not been fully implemented on the airframe program. In addition 
it must be emphasized that a system like WIPICS, if it is to be useful and 
to generate cost savings, must be faithfully used by a large number of people 
in many different shops throughout the NARF. Discussions with the management 
of the NARF seem to indicate that during the time period involved many people 
were not sufficiently knowledgeable to use the system effectively. As a 
result, the accuracy of the data in the system was suspect and the usefulness 
of the system questionable. These conclusions are drawn primarily from infor- 
mation provided by the Naval Area Audit Service. 

The lack of agreement among the various models used illustrates the effect 
of different underlying assumptions and differing model characteristics. There 
is no objective way to choose among these alternative models. However, the 
continuous models do present statistics which enable one to construct confidence 
intervals and test hypotheses about the importance of variables. Thus there 
is some tendency to find the results of these models more believable than the 
results of the linear economic model. Even among the continuous models there 
is disagreement however. The problems here stem mainly from the fact that the 
data are inherently variable. The continuous cost functions which were esti- 
mated while explaining large proportions of the variations in costs are still 
accurate to within only 5 to 10 percent of total costs. Differences among the 
alternative regression analyses are well within these bounds. This emphasizes 
the fact that no definitive conclusions concerning the cost effectiveness of 
the WIPICS system on the airframe program may be drawn at this time. 

The possible sources of this unexplained variation in total costs are 
numerous. Some of them are discussed briefly below. 

Measurement of the variables. As discussed in Chapter Three, the various 
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regression analyses use several proxy variables to measure prices, inputs 
and outputs of the production activity at the NARF, This use of proxy 
variables to measure desired variables will of course inject unknown errors 
into the analyses. In particular the treatment of material cost as a quan- 
tity, thus ignoring price increases for material may have been an important 
source of such errors. This particular source of error was to some extent 
adjusted for by including the production date as a variable in version one 
of the job cost model. 

Collection of the data . Two particular types of data collection pro- 
blems are possible sources of unexplained variation. One problem arises 
from the fact that in some instances production load norm is renegotiated 
after a particular repair job is begun. While the renegotiated norm may 
be more representative of the actual work to be done it seems likely that 
the series of norms some renegotiated and some not, bear a random relation- 
ship to work accomplished. No attempt was made to isolate those jobs with 
renegotiated production load norms for special analysis^ this may be a 
source of unexplained variations. 

Another data collection problem concerns the allocation of material 
cost to jobs and impacts on the assumption that different jobs in the NARF 
are independent of one another. In some cases **job a” needs a part which 
is not available. To expedite the aircraft a used part from **job b” is 
placed on **job a” and a new part ordered for *’job b”. By this process 
”job a” is completed earlier than it would have been and **job b**, which 
has many additional operations to be performed on it, is completed no later 
than it would have been. The problem, from the point of view of data collec- 
tion, is which job gets charged for the new part. Unfortunately, the answer 
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to this question is not uniform. In some cases the job which needed the 
part "job a" is charged for the new part in other cases the job which 
actually received the new part "job b" is charged. This problem impacts 
on man hour data as well. Here the question is which job should be charged 
the labor cost of removing the used part from "job b" and of replacing it 
with the new part. 

The existence of backrobbing tends to argue that jobs in the shop are 
not independent of one another. Although backrobbing may be too infrequent 
to incorporate into the model explicitly, it remains a source of unexplained 
variation. 

Assumptions ^nd form of the relations . Any attempt to describe the be- 
havior of a complex organization with a relatively simple relation involving 
only a few variables is bound to be subject to some error. The relations 
estimated in this report are, of course, not exceptions to this general rule 
and thus some unexplained variation in the data is to be expected. While 
it is not desirable to complicate the relations to the point of explaining every 
detail of the operations there are several influences on production costs which 
may require explicit treatment. The effects of changes in retirement policy, 
the impositions of personnel ceilings, and various admonitions and constraints 
placed on the NARF have been ignored in the cost functions which have been 
estimated. These omissions as well as others are also sources of unexplained 
variation in costs in the analyses. 

Despite the fact that there are many factors left out of the regression 
equations, the fact that variables were measured subject to error and the fact 
that proxy variables were used; each of the continuous cost functions derived 
above explain over ninety percent of variation in total costs of production 
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in the aircraft and engine programs at the NARF. Also, while there are dif- 
ferences in the prediction of total costs of production among the various con- 
tinuous models an analysis of each requires the conclusion that the net cost 
savings of the WIPICS system at this point in time are not significantly 
different from the usual random variations in costs experienced at the NARF. 
This conclusion agrees with other observations on the system provided by 
independent sources • 
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